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Abstract 

This  note  analyzes  the  loss  of  bandwidth  due  to  fragmentation  caused  by  the  variable 
length  packets  in  the  asynchronous  frame  of  a  communication  cycle  in  the  protocol 
used  by  the  Ganglia  network  [Clark  1989].  We  provide  a  simple  back-of-tbe-envelop 
computation,  which  is  later  justified  by  a  more  rigorous  analysis. 
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Section  1  Introduction 


1      Introduction 

Robot  control  systems  typically  consist  of  cooperating  micro-computers.  The  traditional  ap- 
proach of  using  a  shared  memory  on  a  common  bus  for  the  purpose  of  communication  among 
the  micro-computers  has  several  serious  problems.  Such  an  architecture  will  have  difficulty  with 
increasingly  complex  robot  conglomerates,  for  instance  the  ones  occurring  in  dexterous  manipu- 
lation systems  or  intelligent  workcells;  there  are  mechanical  and  electrical  limits  to  the  extension 
of  the  shared  memory  systems;  and  most  importantly,  in  such  a  centralized  system,  the  most 
interesting  and  delicate  signals  have  to  travel  the  farthest  distance.  To  alleviate  these  problems, 
Clark  suggested  a  new  robot  control  architecture  and  communication  protocols  called  Ganglia, 
in  his  dissertation  [Clark  1989].  The  Ganglia  architecture  allows  the  robot  controllers  to  be 
distributed  throughout  the  system,  and  in  fact,  if  each  of  the  controller  could  be  miniaturized 
to  the  size  of  a  match  box,  then  the  components  can  be  physically  distributed  through  out  the 
robot  system.  However,  currently  the  focus  has  been  on  the  communication  problems,  and  on 
the  design  of  the  communication  protocols  that  take  advantage  of  the  periodicity,  regularity  and 
frequency  of  the  messages  such  as:  servo,  steady-state,  changing-state  and  exception  messages, 
occurring  in  the  robot  systems.  A  brief  survey  of  the  real-time  research  work  at  NYU  that  dis- 
cusses these  issues  may  be  found  in  [Clark  et.  al.  1989].  Some  of  the  applications  of  the  suggested 
architecture  may  be  found  in  [Clark  et.  al.  1988],  [Hong  et.  al.  1988]  and  [Mishra  1989].  A  robot 
conglomerate  for  which  the  Ganglia  architecture  is  well-suited  is  the  dexterous  manipulation 
system  at  the  NYU  Robotics  Research  Laboratory  (see  figure  1). 

The  communication  protocol  employed  by  Ganglia  at  any  level  may  be  described  in  terms 
of  a  communication  cycle.  The  cycle  has  three  major  components:  the  servo  cycle  or  servo 
frame,  the  exception  poll,  and  the  asynchronous  cycle  or  asynchronous  frame.  Thus  the  traffic 
is  partitioned  into  three  categories: 

1.  Periodic  Time-Constrained  Messages  include  the  servo  messages  and  some  of  the  steady- 
state  messages.  These  are  transmitted  during  the  servo-cycle. 

2.  Exception  Messages  are  transmitted  in  response  to  the  exception  polls. 

3.  Asynchronous  Messages  include  all  other  messages.  These  are  transmitted  during  the  asyn- 
chronous cycle.  This  category  can  be  further  refined  depending  on  the  time-constraints. 

A  detailed  discussion  of  the  Ganglia  communication  cycle  may  be  found  in  Chapter  7 
of  [Clark  1989].  Figure  2  shows  the  communication  frame.  In  this  scheme,  the  asynchronous 
frame,  which  includes  the  asynchronous  traffic,  may  be  analyzed  using  a  polling  model.  In 
the  asynchronous  frame,  the  control  node  polls  the  nodes  at  the  same  level,  for  asynchronous 
messages,  beginning  where  it  left  off  the  previous  cycle.  The  control  node  can  poll  for  the 
remainder  of  the  frame.  However,  in  order  to  ensure  that  the  next  frame  will  begin  on  time,  the 
controller  must  not  poll  a  node  unless  a  message  from  that  node  will  be  completed  before  the 
end  of  the  frame.  There  is  some  bandwidth  lost  due  to  this  fragmentation.  In  this  short  note, 
we  provide  an  analysis  of  the  expected  value  of  fragmentation  and  a  related  quantity,  spillover. 
The  technique  employed  is  a  fairly  standard  technique  from  probability  theory  and  the  results 
seem  to  be  known  as  folk-theorems. 
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Figure  1:  Utah/MIT  hand  and  PUMA  arm. 

This  paper  is  organized  as  follows:  section  two  describes  the  problem  and  introduces  a 
somewhat  abstract  model.  Section  three  presents  a  heuristic  argument  to  provide  a  back-of- 
the-envelop  computation  of  the  expected  values;  however  these  results  can  be  wrong  in  some 
special  cases.  Section  four  provides  a  rigorous  analysis,  and  derives  the  exact  formulas;  it  also 
provides  the  justification  for  the  results  obtained  via  the  simpler  heuristic  arguments.  The  paper 
concludes  with  an  estimate  of  the  expected  spillover  in  a  typical  Ganglia  implementation  and 
some  discussions  of  the  shortcomings  of  the  analysis. 


2      Spillover  Problem 


Fragmentation  in  Ganglia's  communication  cycles  is  caused  by  the  fact  that  during  a  communi- 
cation cycle  the  control  node  must  stop  polling  for  asynchronous  messages  in  time  to  ensure  that 
it  will  be  able  to  start  the  next  cycle  on  time.  Consider  figure  2,  which  shows  the  communication 
cycle  or  frame.  If  tp  is  the  length  of  a  poll  (this  is  also  the  length  of  a  message  header)  and  tmax  is 
the  maximum  message  size  (header  and  data)  then  the  control  node  must  stop  polling  tp  +  tmax 
seconds  before  the  end  of  the  cycle  to  ensure  that  the  message  in  response  to  the  poll  does  not 
prevent  the  control  node  from  beginning  the  next  cycle  on  time.  The  dashed  line  in  the  figure 
represents  the  point  tp  +  tmax  seconds  before  the  end  of  the  cycle.  The  fragmentation  in  each 
cycle  is  not  equal  to  tp  +  tmax,  however,  since  a  message  or  poll  initiated  before  the  dashed  line 
may  continue  transmitting  into  the  interval  between  the  dashed  line  and  the  end  of  the  cycle. 
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Figure  2:  Communication  Frame. 

Thus,  the  fragmentation  in  each  cycle  is  reduced  by  the  amount  that  the  last  message  in  the 
cycle  falls  into  this  interval.  Calculating  the  amount  of  fragmentation  thus  becomes  the  problem 
of  calculating  the  amount  that  the  last  message  in  each  cycle  "spills  over"  into  this  interval. 

We  will  begin  with  a  more  abstract  problem.  Consider  a  bin  full  of  sticks.  The  sticks  are 
each  of  integral  length  with  the  maximum  length  of  k  units.  We  will  draw  sticks  from  the  bin 
at  random  —  one  stick  at  a  time.  The  supply  of  sticks,  for  our  purposes,  is  inexhaustible  so 
that  finding  the  bin  empty  is  never  a  problem.  Now,  consider  a  line  on  the  floor  starting  at  the 
wall  and  terminating  at  a  mark  which  is  N  units  from  the  wall.  Now,  suppose  we  draw  sticks 
from  the  bin  and  place  them  end-to-end  along  the  line  starting  at  the  wall  and  stop  when  a 
stick  covers  the  mark.  The  problem  of  message  spillover  corresponds  directly  to  the  problem  of 
determining  how  far  the  last  stick  extends  beyond  the  mark.  The  reader  should  note  that  to  be 
consistent  with  the  original  problem  if  the  length  of  the  sticks  is  exactly  N  then  one  more  stick 
is  drawn  and  added  to  the  line.  In  other  words  "covering  the  mark  at  TV"  means  that  the  point 
N  +  1  units  from  the  wall  is  reached.  Figure  3  shows  the  situation. 


3      Expected  Spillover:  Heuristic  Computation 

We  begin  with  some  heuristic  computation  for  the  expected  value  of  the  spillover.  The  com- 
putation provides  an  easy  way  of  determining  the  expected  value,  but  the  justification  for  the 
method  will  have  to  appeal  to  certain  'ergodicity'  properties.  In  the  following  section,  we  provide 
a  rigorous  derivation  of  the  correct  formulas  for  the  expected  spillover. 

Let  us  illustrate  this  technique  with  the  simplest  case,  where  a  stick  of  each  length  between 
1  and  k  may  be  chosen  independently  with  equal  probability.  Since  we  are  interested  in  the 
asymptotic  behavior,  we  shall  assume  that  TV  is  much  larger  than  k.  Now,  if  a  sequence  of  sticks 
is  chosen  until  the  total  length  of  the  sequence  exceeds  N,  then  there  will  be  approximately 
the  same  number  (say,  L)  of  sticks  of  each  size  in  the  sequence,  since  lengths  of  the  sticks  are 
assumed  to  have  a  uniform  distribution. 
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Figure  3:  Simplified  "spillover"  problem. 


Thus, 


V-      T  M  AT  2N 

>    i  •  L  ss  N     and     L  ss  — — . 

If  we  consider  only  sticks  of  length  i  then  they  will  cover  approximately  i  ■  L  units  of  the  line, 

and 

.       i  ■  L  2i 

Prfa  stick  of  length  t  covers  the  mark  at  N]  s»  —77-  =  -rr, 77- 

1                       fe                                               J        N  k(k+l) 

Thus  the  expected  length  of  the  stick  covering  the  mark  at  N  is  roughly: 

V — - — i  =   — - — Ti2 

2         k(k  +  l)(2Jb+  1) 
k(k+l)  6 

2fc  +  l 
3      ' 

Thus  the  expected  spillover  is  about  fc/3  units. 

The  above  argument  can  be  generalized  to  the  situations,  where  lengths  of  the  sticks  have 
an  arbitrary  distribution,  given  by: 

Pi  =  Pr[a  stick  of  length  i  is  chosen],     where  t  =  1, . . .,  k. 

Let  /is  and  a2s  be  the  mean  and  variance  of  this  distribution,  respectively. 

As  before,  a  sequence  of  sticks  is  chosen  until  the  total  length  of  the  sequence  exceeds  iV, 
where  TV  >  k. 
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Let  Li  be  the  number  of  sticks  of  length  i  occurring  in  the  sequence,  then 


and  for  each  i 
Combining  these,  we  get 


fc 


L, .&pl(L1  +  L2  +  ---  +  Lk). 


k 

1=1 

k 

*    ^2i,Pi(Li  +  L2  + 1-  Lk) 

1=1 

=    /i5(Ii  +  X2  +  •  •  ■  +  Lk). 

Proceeding  as  we  did  in  the  previous  section,  we  get 

i  ■  L 
Pr[a  stick  of  length  i  covers  the  mark  at  N]  «  — — - 

_     ipt(Li  +  L2+  ■  ■■  +  Lk)  _  tp< 
Hs{Li  +  L2  +  •■•+  Lk)       us' 

and  the  expected  length  of  the  stick  covering  the  mark  at  N  is  roughly: 

*    z^p,     =     E[S2} 
~{  US  MS 

Ms  +  ^I 

Ms 

=     MS  +  ^£- 
Ms 

Thus  the  expected  spillover  is  about  Ms/2  +  a|/2/is- 

We  note  that  the  above  heuristic  argument  yields  a  result  that  may  be  false  in  general;  for 
instance,  consider  a  distribution  in  which 

P\  =  P2  =  ■■■  =  Pk-i  =  0,     and     pk  =  1. 

Above  calculation  results  in  an  expected  spillover  value  of  k/2.   This  result  is  false,  since  it  is 
easy  to  see  that  the  spillover  is  exactly  k  -  7,  where  7  =  remainder(Ar,  k). 
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4      Expected  Spillover:  Rigorous  Computation 

Let  us  assume  that  the  lengths  of  the  sticks  have  an  arbitrary  distribution,  given  by: 

Pi  —  Pr[a  stick  of  length  i  is  chosen],     where  i  =  1, . . . ,  k. 

Let  p.s  and  as  be  the  mean  and  variance  of  this  distribution,  respectively.  We  further  assume 
that  for  the  indices  i\  <  i?  <  •  •  •  <  J/, 

P.,  >  0,p,2  >  0,...,p„  >  0, 

and  all  other  p^'s  are  zero.  Also,  let 

m  =  GCD(t'i,i2, . .  .,i(). 

Thus  there  are  integer  coefficients  a,\,  a?,  .  ■  .,  a/,  such  that 

va  —  a\  •  i\  +  a-i  •  t'2  +  •  •  •  +  a/  •  t;. 

Let  Un  denote  the  probability  that  a  sequence  of  sticks  chosen  under  the  above  distribution 
has  a  total  length  equal  to  JV,  where  N  >  k.  Then,  we  have 

Un  =  Pi  •  Un-i  +  P2  •  Un-2  + \-  Pk  ■  ^N-fc- 

The  characteristic  equation  for  the  above  condition  is  then  given  by 

xk-p1xk-l-p2xk-2 pfc  =  0.  (1) 

Lemma  4.1   Let  X  be  a  (complex)  root  of  the  characteristic  equation  (1).   Then 

1.  |A|  <  1,  and 

2.  x/|A|  =  1  then  X  =  u>m  is  an  mth  root  of  unity. 

PROOF. 

1.  Since  A  is  a  root  of  the  characteristic  equation,  we  have 

A*  =p1Afc-1+p2A*-2  +  ...  +  pt, 
and 

|A|fc     =      \plXk-1-rp2Xk-*  +  ...  +  pk\ 

<    Pi\X\k-1+p2\X\k-9  +  —  +  Pk. 

Since  the  right  hand  side  of  the  preceding  equation  is  a  convex  combination  of  |A|fc_1, 
|A|fc~2,  . . .,  1,  it  is  bounded  from  above  by  max(l,  |A|fc_1),  and 

|A|*  <  max(l,  lAI*"1). 

Thus 

|A|<1. 
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2.  Assume  that  |A|  =  1.  Then, 

Afc  =  Pil  A*"*'  +  Pi7  A*-<3  +  •••  +  ?„  A*"" . 

Thus,  in  the  complex  plane,  the  point  corresponding  to  Xk  is  in  the  interior  of  the  convex- 
hull  of  the  points  corresponding  to  the  points  A*-'1,  Afc_'2,  . . .,  A*-*'.  Since 

|  A*— |  =  |Afc-*'J|  =  .-.=  |A*-"|  =  1, 

we  conclude  that  if  A*-'1 ,  A*-'2 ,  . . .,  A*-1'  are  not  all  equal,  then  A*  belongs  to  the  interior 
of  the  unit  disk  about  the  origin  and 

|Afc|  =  |A|*<  1, 

which  contradicts  the  initial  hypothesis  that  |A|  =  1.  Indeed, 

\k  =  A*-1  =  A*-2  =  ...  =  Afc-\ 

and 

A°  =  A*1  =  A'2  =  •••  =  A"  =  1. 

Thus 

Am  =  \(ai>i+<*-2>7-i hO|«i)   _   j  ;.         yn  _  j  _  q 

Furthermore,  (xm  -  1)  is  a  factor  of  the  characteristic  polynomial  1.  Thus 

A  =  u;m  =  m'    root  of  unity.      □ 

It  is  easy  to  see  that  the  polynomial  corresponding  to  the  characteristic  equation  is  square 
free,  and  each  root  occurs  with  multiplicity  one.  Thus  the  solution  to  the  recurrence  equation 
given  in  the  beginning  of  the  section  is: 

Us  =  Ki  +  K*£  +  K&,™  +■■■  +  KmLJ^-1)N  +  Km+1r%+,  +  ■  ■  ■  +  A'*r£\ 

where  rm+1  <  1,  . . .,  rjt  <  1.  Hence, 

lim   UN  =  K1  +  A2a£  +  K*>™  +  •■•  +  KmJ^N . 

N— »oo 


'The  characteristic  polynomial  for  the  recurrence  equation  can  be  factored  as  follows: 

i 

=     i*-''(zm-l) 


k  k—\.  k  —  \2  Jfc—  ti 

x    -pHx  -p„x         p,tx       ' 


[(i',-m  +  i,'-2m  +  .-  +  r''-') 

+  (1  -  p., )  (x"-"-m  +  i"-"-2m  +  . . .  +  ««•-'") 

+  (1  -  P.,  -  P>3)  (x'>-3-m  +  x»-"-2m  +  ...  +  ,<•--.) 

+  •■• 

+  (l-Pll p,,.,)   (!"-"-'-"*  +l''-"--2m+...+  l)]. 
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Using  the  fact  that,  for  all  TV  not  divisible  by  m,  Us  =  0,  we  see  that 

A  i  =  A'2  =  •  •  •  =  A  m  =  — , 

m 

and 

lim^  =  §(l  +  ^  +  ^  +  ---  +  ^-1)'V)- 
S—>oo  m  \  ' 

Now  let  us  return  to  the  original  problem,  and  make  a  distinction  between  two  concepts: 
excess  and  spillover. 

1.  Assume  that  a  sequence  of  sticks  is  chosen  until  the  total  length  of  the  sequence  is  >  N . 
Then  the  random  variable  excess  denotes  the  amount  by  which  the  total  length  of  the 
sequence  of  sticks  exceeds  N. 

2.  Assume  that  a  sequence  of  sticks  is  chosen  until  the  total  length  of  the  sequence  is  >  N . 
Then  the  random  variable  spillover  denotes  the  amount  by  which  the  total  length  of  the 
sequence  of  sticks  exceeds  N . 

Observe  that 

^[spillover]  —  ^[excess]     =     Pr[excess  =  0]  /]i  ■  p, 

=    Us  ■  vs- 

Now  let  us  assume  that  N  =  (5m  -  a  =  8m  +  7,  for  some  0  <  0,7  <  m,  and  for  sufficiently 
large  values  of /?  and  6.  Thus,  7  =  remainder(Ar,  m). 
It  is  immediately  seen  that 

Pr[excess  =  a  +  (i  —  l)m] 

=       Us-m+a  Pirn  +  Us-2m+a  P(i+l)m  + 1"  l?S -([k/m\+l-i)m+a  P[k/m]m 

=       [Pirn  +P(,+i)m  +  •••  +  P[A:/mJm)  C, 

where  1  <  i  <  [^-J .  For  all  other  values  of  "excess,"  the  probabilities  are  zero. 
Thus, 


and  C  =  m/fis- 
Hence 


Cy\i  •  p,m  =  —  y"(im)p,m  = =  1, 


lim   Us  =  l  +  ^+^m    +•••  +  <*» 

S— 00  fiS 


We  are  now  ready  to  evaluate  the  expected  value  of  excess,  which  is  given  by 

£[excess]     =     ]P  Pr[excess  =  a  +  (i  -  l)m](a  +  (i  -  l)m) 
1=1 

-     —  ^p,m(2ia  +  i2m  -  im) 


Section  4  Rigorous  Computation 


=  -^vi  ( — ]Oim)P'm  + — y2(im)2p^  -  Y\(im)pim) 

2  * — '  \  m  "—^  m  "-^  *-^  J 

m    I  2a  u%  +  a%  \ 

=     7—     —Ms  +  — -  ns 

2/is  \rn  m  J 

ns        o%        m 
=     —  +  —s-  -  —  +  a. 
2    T  2/x5       2  T 

Thus  the  expected  value  of  spillover  can  be  evaluated  as  follows: 
Case  1:  q  /  0. 
Then  Un  =  0,  and  7  =  m  —  a  ^  0.  In  this  case  the  expected  value  of  the  spillover  is  given  by: 

2 

^[spillover]     =     —  +  77^-  -  —  +  a  +  UNns 
2        ins        2 

2    +  2ns       2       7- 

Case  2:  o  =  0. 

Then  £//v  =  m/  ns  ^  0i  and  7  =  0.  In  this  case  the  expected  value  of  the  spillover  is  given  by: 

lis  <7c  T7l 

E [spillover]     =     —  +  — *-  -  —  +  a  +  Usns 
2        2ns        2 

Us        g|        m 

~2  +  2ns  +  "2" 

2    +  2ns       2       7' 
The  above  results  can  be  succinctly  summarized  in  the  following  theorem: 

Theorem  4.2    Consider  an  experiment  in  which  a  sequence  of  sticks,  each  of  length  between  1 
and  k,  is  chosen,  with  the  stick  of  length  i  occurring  with  probability  p,,  until  the  total  length 
of  the  sequence  strictly  exceeds  N.    We  assume  that  N  3>  k.   Let  the  random  variable  spillover 
denote  the  amount  by  which  the  total  length  of  the  sequence  exceeds  N. 
Then  the  expected  value  of  the  spillover  is  given  by 

£[spillover]  =  ^  +  ^  +  !-7,  ■     (2) 

where 

m  =  GCD{i  I  1  <i<k  and  pt  >  0}, 

7  =  remainder(Ar,  m),  ns  ,s  "*e  mean  and  as  is  the  variance  of  the  probability  distribution  of 
the  lengths  of  the  sticks.        □ 

Now  let  us  reconsider  the  case  when  the  lengths  of  the  sticks  have  a  uniform  distribution  in 
the  range  [l,k].  Thus  m  =  1,  and  7  =  0.  Furthermore, 

k  + 1         ,       2      k2  - 1 


ns  =  — r—     and     as  = 
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Thus, 

r     „        ,  it  +  1       it  -  1       1 

^[spillover]     =     ^-  +  -^-  +  2 

k  +  2 

3 

In  general,  if  the  lengths  of  the  sticks  have  some  arbitrary  distribution  in  the  range  [l,k], 
but  p,  >  0  for  all  i  e  [l,k]  then  m  =  1,  and  7  =  0.  Thus, 

^[spillover]  =  —  +  —  +  -, 

where  us  is  the  mean  and  as  is  the  variance  of  the  probability  distribution.  Note  that  this  result 
differs  only  by  an  additive  term  of  1/2,  from  the  formula  derived  by  the  informal  argument  in 
the  preceding  section. 

Let  us  now  reconsider  the  troublesome  distribution  of  the  previous  section; 

Pi  =  P2  =  ■••  =  Pfc-i  =  0,     and     pk  -  1. 
In  this  case  m  =  k,  fis  =  k  and  as  =  0.  Thus, 

it       it 
E  [spillover]     =     -  +  -  -  7 


2       2 
fc-7, 


where  7  =  remainder(Ar,  k). 


5      Message  Fragmentation 

In  the  previous  section,  we  developed  a  formula  for  the  expected  spillover  in  the  abstract  problem 
of  placing  sticks  on  a  line.  The  end  result  is  equation  (2)  of  Theorem  4.2.  Messages  in  Ganglia 
have  a  form  which  allow  us  to  further  refine  the  estimate  of  the  spillover. 

In  the  communication  cycles  the  "stick"  corresponds  to  the  poll- response  sequence.  During 
the  asynchronous  frame  the  control  node  transmits  a  poll  message  which  is  tp  bytes.  The  polled 
node  must  respond  with  a  message  whose  header  is  also  tp  bytes  long  and  whose  data  portion, 
to,  is  zero  to  Dmax  bytes  long  (where  Dmax  >  0  is  the  maximum  size  for  the  data  portion  of 
a  message  which  is  equal  to  tmax  -  tp).  It  can  be  reasonably  assumed  that  each  message  may 
occur  with  some  non-zero  probability  in  our  application;  hence,  m  =  1  and  7  =  0.  Thus,  the 
"length"  of  the  stick  is  ts  =  2tp  +  to,  where  tp  is  a  constant  and  0  <  to  <  Dmax- 

To  calculate  ^[spillover],  we  need  /15  and  os.  The  mean,  us  =  E[t$],  is  simply  2tp  +  E[to] 
or 

Us  =  2tp  +  /i£),     since  tp  is  a  constant. 

The  variance  as  is  E[ts]  -  /j.s  where 

E[t*s]    =    E{(2tp  +  tD?) 

=     4tl  +  4tpliD  +  E[t2D). 
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Substituting  and  simplifying  we  get 

aS  -  £[*£>]  -  f*D i  =  aD- 
And  finally, 

^spillover]     =     f  +  ^  +  I 

'p+2+    2    +4<p  +  2^-  (3) 

In  a  typical  Ganglia  implementation,  the  header  is  tp  —  32  bytes  long.  For  purposes  of 
analysis,  we  will  assume  the  data  portion  of  the  messages  is  exponentially  distributed  with  a 
mean  length  of  hd  =  32  bytes.  Thus 

Ms  =  96  bytes        and      a\  =  a%  =  (32)2  =  2048, 

and  we  get  ^[spillover]  =  323/6  =  53.833  bytes. 

Two  further  notes  should  be  made.  Firstly,  in  a  Ganglia  system,  the  lengths  of  the  asyn- 
chronous frames  are  not  all  the  same  (i.e.,  N  is  not  a  constant).  This  adds  another  random 
variable  to  the  analysis.  However,  there  are  a  small  number  of  different  cycle  lengths.  So  if  de- 
tailed analysis  were  desired  for  a  particular  system  the  analysis  could  be  applied  to  each  frame 
size  separately.  Secondly,  in  Ganglia  systems,  it  is  not  clear  that  N  is  large  (an  assumption 
that  makes  the  above  derivations  go  more  smoothly).  If  one  considers  the  "maximal  system" 
example  discussed  in  Clark's  thesis  [Clark  1989],  it  is  quite  possible  that  during  some  cycles 
there  is  only  enough  time  for  one  or  two  polls  during  the  asynchronous  frame,  thus  violating 
the  assumption  that  N  >  2fp  +  Dmax.  However,  in  the  absence  of  real  data  and  lacking  more 
powerful  analysis  tools,  these  questions  remain  open. 
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